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Abstract. We consider a controlled-diffusion process pertaining to a chain of 
distributed systems with random perturbations that satisfies a weak Hormander 
type condition. In particular, we consider a stochastic control problem with 
the following objectives that we would like to achieve. The first one being of 
a reachability-type that consists of determining a set of attainable distribu¬ 
tions at a given time starting from an initial distribution; while the second 
one involves minimizing the relative entropy subject to the initial and desired 
final attainable distributions. Using the logarithmic transformations approach 
from Fleming, we provide a sufficient condition on the existence of an optimal 
admissible control for such a stochastic control problem which is amounted to 
changing the drift by a certain perturbation suggested by Jamison in the con¬ 
text of reciprocal processes. Moreover, such a perturbation coincides with a 
minimum energy control among all admissible controls forcing the controlled- 
diffusion process to the desired final attainable distribution starting from the 
initial distribution. Finally, we briefly remark on the invariance property of 
the path-space measure for such a diffusion process pertaining to the chain of 
distributed systems. 


1. Introduction. The problem of forcing a diffusion process to a final attainable 
configuration starting from an initial distribution has been of particular interest 
from both physical and mathematical points of view (e.g., see [ 27 ] for the original 
formulation of this problem). Notably, this problem and its variants have been 
widely studied in the literature (e.g., see [21], [20] [22] or [23] in the context of 
stochastic optimal transportation problem; see also [24] and [32] in the context of 
variational characterization of Schrodinger processes; and [13] or [7] in the context 
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of controllability of systems governed by parabolic PDEs). The setting of these 
papers, which is the rationale behind our approach, consists of constructing a class 
of stochastic processes (i.e., a reciprocal process along the works of [16], [17] and [4]) 
in connection with stochastic optimal control theory with respect to the desired end¬ 
point distributions or in connection with that of the problem of minimizing relative 
entropy subject to the controlled and reference diffusion processes (e.g., see [20], [22] 
or [7] for additional discussions). Here, our main interest is to throw some light on 
the structure of the controlled-diffusion process pertaining to a chain of distributed 
systems and, at the same time, clarifying questions concerning minimization of 
relative entropy subject to the initial and desired final attainable distributions for 
such a controlled-diffusion process.^ 

In this paper, we specifically consider the following distributed system, which is 
formed by a chain of n subsystems (where n > 2), with a random perturbation that 
enters only in the first subsystem and is then subsequently transmitted to other 
subsystems, i.e., 

dxl = mi(t, xj,..., x^)dt + a{t, x^,..., x^)dWt 
dXf = m 2 (t, Xf,..., x^)dt 

dx^ =m3(t,xf,...,x^)dt I 

dx'^ = mn{t,Xf~^ ,x'^)dt, 0<t<T 

where 

• a;* is an R'^-valued state information for the ith subsystem, with i £ {1, 2,..., n}, 

• mi: M+ X -> and rrij : R+ x for j = 2,..., n, are 

bounded continuous functions and satisfy appropriate Holder conditions, 

• a: K+ X —>■ is a bounded continuous function, with the least eigen¬ 

value of crcr^ uniformly bounded away from zero, i.e., 

a{t,x^,... ,x^) (t,x^,... ,x^) h ^Idxd, V(a;^,...,x”) £ R”‘^, Vt £ R+ 

for some A > 0, 

• Wt (with Wo = 0) is a d-dimensional standard Wiener process. 

Note that such a chain of distributed systems has been well discussed in various 
applications (e.g., see [6], [1], [8] and [28] and the references therein). For example, 
when n = 2, the equation in (1) can be used to describe stochastic Hamiltonian 
systems (e.g., see [6] or [28] for additional discussions). 

Next, let us introduce the following notation that will be useful later. We use 
bold face letters to denote variables in R”"^, for instance, 0 stands for a zero in R"'^ 
(i.e., 0 £ R"'^) and, for any f > 0, the solution (^xl,Xf ,..., x") to (1) is denoted by 
xt. Moreover, for (t, ..., x")) £ R+ x R(’^“t-i- 2 )d^ j = 2,..., n, the function 

x^ mj[t,x^~^,... ,x'^) is continuously differentiable with respect to x^ and its 
derivative denoted by ... ,a;") >->■ D^jmj{t,x^~^,... ,x”). 

Then, we can rewrite the stochastic differential equation (SDE) in (1) as follow 

dxt = M(t, Xt)dt -I- Ga{t, xt)dWt, (2) 

where M = [mi, m 2 ,..., m„] is an R”‘^-valued function and G = [id, 0,..., O] ^ 
stands for an {nd x d) matrix that embeds R"^ into R"'^. Moreover, the infinitesimal 

^In this paper, our intent is to provide a theoretical framework, rather than considering a 
specific numerical problem or application. 
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generator associated with (2) is given by^ 

2 _ n 

+mi(t,x) ■ D^i + V (3) 

Z 3 —^ 

where a{t,x) = (T(t, x)cr^(t, x). 

Remark 1.1. Note that, in (1), the random perturbation enters only in the first 
subsystem through its diffusion and is then subsequently transmitted to other sub¬ 
systems through their respective drift terms. As a result, such a chain of distributed 
systems is described by an -valued diffusion process xj, which is degenerate in 
the sense that the second-order operator associated with it is a degenerate parabolic 
equation. Moreover, we assume that the distributed system in (1) satisfies a weak 
Hormander type condition (e.g., see [15] or [10, Section 3] for additional discus¬ 
sions). 

Throughout this paper, we assume that the following statements hold for the 
distributed system in (2) (or (1)). 

Assumption 1.2. 

(a) The functions mi(t,x) and mj{t,x.N^') for j = 2, ...,n satisfy appropriate 

Holder conditions with respect to x and respectively. Moreover, a{t,x.) 

is a bounded C'^([0,T] x -function; a{t,x.) and Dj.ia{t,x.) are bounded and 
satisfy appropriate Holder conditions with respect to both x and t. 

(b) The infinitesimal generator Ct,x is hypoelliptic (e.g., see [15] or [10]/ 

Remark 1.3. In general, the hypoellipticity assumption is related to a strong ac¬ 
cessibility property of controllable nonlinear systems that are driven by white noise 
(e.g., see [30] concerning the controllability of nonlinear systems, which is closely re¬ 
lated to [29]; see also [10, Section 3]/ Moreover, the Jacobian matrices D„.imi(t,x) 
and D,^j-imj{t,x.N^'j for j = 2,... ,n are assumed to be nondegenerate, uniformly 
in time and space. Note that the hypoellipticity assumption also implies that the 
diffusion process xt has a transition density with a strong Feller property. 

Remark 1.4. Here, it is worth mentioning that there are some results, based on 
Malliavin calculus under strong Hormander conditions, that provide a sensitivity 
measure of the system with respect to noise, where a nonzero sensitivity condition 
(which is summarized by a nondegenerate Malliavin matrix) is used in verifying the 
existence of such a density function (e.g., see [18] and [25] for additional discus¬ 
sions). 

The paper is organized as follows. In Section 2, we provide some preliminary 
results. Section 3 formally states the stochastic control problem considered in this 
paper. In Section 4, using the logarithmic transformations approach from Fleming, 
we provide a sufficient condition on the existence of an optimal admissible control 
for such a stochastic control problem. This section also contains additional results 
concerning minimization of relative entropy subject to controlled and reference dif¬ 
fusion processes. Finally, in Section 5, we briefly remark on the invariance property 
of the path space measure for such a diffusion process pertaining to the chain of 
distributed systems. 


for j = 2,... n. 
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2. Preliminaries. In this section, we provide some preliminary results that will be 
useful later in Section 4. Note that, for any square integrable ]R'“^-valued random 
variable ^ that is independent to {Wt, 0 < t < T}, the SDE in (2) admits a weak 
solution in [0, T) (i.e., in [0, T — e] for any e > 0) with initial condition Xq = 

Moreover, the fundamental solution (i.e., the transition density) ( 7 (s,x, t,y) for 
0 < s < t and x, y £ of the PDE of parabolic type satisfies the following 

^q(t,x,r,y) + A,x(?(i,x,r,y) = 0 in [0,r)xR’"‘^ 

limg(t,x,r,y)=<5y(x) for x,yGR"'' (4) 

and it is twice continuously differentiable with respect to x and continuously differ¬ 
entiable with respect to s. Note that, for a fixed arriving point (T, C) £ [0, oo) 
we can approximate the boundary condition in (4) using a sequence of positive func¬ 
tions (<)>e)e>o on that weakly converge towards the Dirac function i5^. 

To this end, we assume that {<j)e)e>o (on the whole satisfies the following 

3e(,>o such that lim sup sup ^!)e(x) = 0. (5) 

c—>-oo 0 <£<ej, |x|>c 

Then, we can approximate the transition density function by 

/ie(t,x) =Et,x{</>s(C)}, V£>0, V(t,x) £ [0,T-e] xR"^ (6) 

where such an approximation also satisfies the following Cauchy problem 

^^^(Cx)-t-£t,x/ie(i,x) = 0 in [0,T-e] X R"'^, (7) 

with boundary condition /ie(T — e,x) = for x £ R”'^. Note that, since q is 

continuous, we have the following 

lim h£(0,x) = limEo,x{?5'E(C)} 

£->■0 £->.0 ^ ^ 

= linr [ ())e(y)9(0, :si,T -e, y)dy 

^-*■0 JRr.d 

= 9(0,x,r,C). (8) 

If we introduce the following logarithmic transformation (e.g., see Fleming [11] or 
[12] for such transformations in the context of stochastic control arguments) 

Je{t,x) =-\oghs{t,x.), (t,x) £ [0,r - e] X R”"^, (9) 

then it is easy to show that x) satisfies the following nonlinear parabolic equa¬ 
tion 

^Je(t,x) -b£t,x./£(Cx) = (10) 

in [0, T — e] x R""^ with the following boundary condition 

Je(r - e,x) =-log^e(x), X £ R”'*. (11) 

Furthermore, Je(t, x) is a value function to the following stochastic control problem^ 

influx I ^ - log(/)e(xT-e) I (12) 
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subject to a controlled version of (2), i.e., 

dx“ = + Gut)dt + Ga{t,x^)dWt, 


Xn = X, 


(13) 


where {ut)o<t<T-e is an K.'^-valued progressively measurable process satisfying 

.T-e 

E / 


111 112 , 

77 Ut _idt < +00. 


Note that, for a given {t, x) € [0, T — e] x the infimum in (12) is achieved when 


= a{t,K)D^i log/ie(t,x). 


(14) 


Later, in Sections 3 and 4, we consider a stochastic control problem, where the 
objective is to force the controlled-diffusion process x“ to the desired final attainable 
distribution starting from the initial distribution using minimum energy control. 

In what follows, let g{x.) be any positive measurable function that satisfies 



q{0,:K,T,z)g{z)dz < +oo for some x S 


Then, the function 


h{t, x) 


q{t,x,T,z)g{z)dz 


(15) 


(16) 


belongs to Cj’^([0,r] x R”"^) and it is also the kernel of the operator (^d/dt + 
i.e, dh{t, x )/dt + Ct^xh{t, x) = 0 in [0, T) x 

Note that an absolutely continuous change of measure on the path-space is related 
to changing the original drift term of the diffusion process associated with the SDE 
in (2) (see also [14]). A particular case was considered in [17] (cf. [17, Theorem 2]) 
leading to the following result. 


Proposition 2.1. Suppose that x* is a weak solution of (2) in [0,T). Let h(t^x.) G 
Cj,’ ([0,T] X R""^^ be a strictly positive solution to following 

=0 in [0,T)xIR'''^ (17) 

ot 

such that E{/i(t,x)} < -Poo and h{s,x) = Es_x{/i(t, x)} for all 0 < s < t < T. 
Then, the following SDE 

dx^ = (M(t, xj*) -P G ait, x[*)T> 3 .i log h{t, x[’')^ dt + Gait, x^)dWt (18) 

admits a weak solution in [0,T). Moreover, if there exists a positive measurable 
function g{x) such that 

/i(s,x) = E^,x{ff(xT)}, (19) 

then the transition density corresponding to (18) is given by 

q^{s, X, t, y) = q{s, x, t, y) . (20) 

his, X) 

Proof. Let (0,P, be the probability space in which the weak solution x (i.e., 
a continuous process) of (2), with initial condition Xq, is defined on. Then, let us 
introduce the following nonnegative martingale process 

hit,xt) 

h(0,xo) 


zt = 


( 21 ) 
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with respect to the natural filtration = <j{xs\0 < s < . 

Note that E{zt} = 1, then we can introduce the following change of probability 
measures on 


dQ 

dP 


= ZT-e 


( 22 ) 


for any fixed e > 0. 

Let / S and 0<s<t<T — e,we have 

IE^x{/(xt)} -/(xs) 


1 


Es.xj^ h{T,yir)Cr,^fi:x:T)+a{T,yir)D^i\ogh{T,yir)-Da:if{xr) drj 
E,,.x<l / h{T,y:r) mi{T,Xr) ■ D^if{y:r) + K- 


h{s,Xs) 

1 

h(s,Xs) 

1 

TT \ 

/i(5,Xs) 

+ i tr(a(T, Xr)Dlif{Xr)) + a{T, Xr)D^l log h{T, Xr) ■ D^lf{Xr) rfuj 

E 


h(s,Xs) 


+ 


1 


,,^[h{t,Xt) mi{T,Xr) ■ D^lf{Xr) + ^)-D^if{Xr) 

tT{a{T,Xr)Dlif(Xr)) + a{T, Xr) D log h{T,Xr) ■ D^lf{Xr) drj 


= lE^x{/ rni{T,Xr) ■ D^lf{Xr) +^_^^mj{T,xl ^)-D^jf{Xr) 


+ {a{T,Xr) Dll f{Xr)) + a{T,Xr)D log h{T,Xr) ■ Dr;lf{Xr) rfuj, (23) 

where we have employed the Ito’s rule for h{t,xt)f{xt). This means that the law 
of X., as a process defined in (ff, Q, ^), solves the martingale problem for 

A'^.x = mi{t,Xt) ■ D^I + mj{t,xl~^) ■ D^i + - tr{a{t,Xr)Dli) 

+a{t,Xt)D^ilogh{t,Xt) ■ D^i (24) 

in [0, T—e]. This is equivalent to saying that (18) has a weak solution x^ in [0, T—e], 
Note that / has compact support and if h{t,Xt) = Es^x{ff(xT)}- Then, we can 
define dQ/dP = (?(xt)- As a result, we obtain the following 

lE^x{/(xt)}-/(Xs) =E^^|^ mi{T,Xr) ■ D^lf{Xr) 

+ '^_^^mj{T,xl-^)-Dr:3f{Xr) + - tl {a{T, Xr) DIi f {Xr)) 


+ a{T,Xr)D^i logh(T,Xr) ■ D^if{xr) drj. 


(25) 


since Xt —>■ xt a.e., we can let f —^ T and conclude by the Lebesque’s dominance 
convergence theorem (see [26, Chapter 4]). 

In order to show (20), we only need to check the following condition 


E?x{/(xt)}=y q^{s,x,t,y)f{y)dy 


(26) 
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for any / G That is, 

g(s,x,t,y)/i(t,y)/(y)dy 


1 


/l(s,Xs) 

= J 9 '‘(s,x,t,y)/(y)dy. 
This completes the proof of Proposition 2.1. 


(27) 

□ 


Next, we state the following proposition (without proof) which is a version of 
the result given in [16] (cf. Beurling [4]). Later, we use this proposition for proving 
Propositions 4.2 and 4.4 in Section 4. 

Proposition 2.2. (cf. [16, Theorems 2.1 and 3.1]^ Let fig and be two probability 
measures on Suppose that q{s,x,t,y), for 0 < s < t < T and x, y G is a 
transition density. Then, there exists a unique pair of a-finite measures {vo,vt) on 
such that the measure fi on x which is defined by 


t{E)= f g(0,x,T,y)uo(c?x)uT(dy), 

J E 


(28) 


has marginals qiQ and (where E is an arbitrary x W^'^-Borel set). Further¬ 
more, vq <C ^0 oTid vt fJ.T mutually absolutely continuous measures. 

Recall the following definition that will be useful later. 

Definition 2.3. Assume that V 2 and ui are a-finite measures defined in the same 
measure space. Then, the relative entropy of V 2 with respect to vi is defined by 


H{i'2Wi) = 


{ MKf) 


dv2, 


+ 00 


if V2 < Ul, 
otherwise. 


(29) 


3. Statement of the problem. Let us consider the following controlled-diffusion 
process 

dx“ = (M{t, x“) + Gut) dt + Ga{t, x“)dlTt, (30) 

where Ut is an admissible control that satisfies 

(i) Ut is an R'^-valued process with measurable sample paths satisfying nonantic- 
ipatory condition, i.e., (Wt — Wg) is independent of Ur, for r < s < t; 

(ii) (30) admits a weak solution in [0, Tj; and 

(iii) E/g ||ut||^_idt < +00. 

Assume that we are given two probability measures qiQ and fix, then we specifi¬ 
cally consider the following problem (which was originally formulated by Schrodinger 
in [27], albeit in a slightly different context). 

Problem 1. Find an optimal admissible control Ut such that 

(a) Xg and xlf are distributed according to fiQ and fix, respectively; and 

(b) Ut (among all admissible controls satisfying (i)-(iii)) minimizes the following 
cost functional 
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In the following section (cf. Propositions 4.2 and 4.4), we provide a sufficient 
condition on the existence for such an optimal admissible control (i.e., a minimum 
energy control) for the above problem. 


4. Main results. In this section, we present our main results - where we first char¬ 
acterize the set of attainable distributions with respect to the class of admissible 
controls mention above in Section 3. Then, we provide a condition on the existence 
of an optimal admissible control forcing the controlled-diffusion process to the de¬ 
sired final attainable distribution starting from the initial distribution and, at the 
same time, we make connections to the problem of minimizing relative entropy 
subject to these end-point distribution constraints.^ 


4.1. The set of attainable distributions. Here, we provide a result that char¬ 
acterizes the set of attainable distributions for the controlled-diffusion process x“ 
in (30) with respect to the above class of admissible controls. 

In what follows, we assume that Xj is a weak solution in [0, T) to the SDE in (2), 
i.e., 


dxt = M{t,Xt)dt + Ga{t,Xt)dWt, xq = .f, 

where the initial point Xq = ^ is distributed according to hq and satisfies ]E|^P < 
- 1 - 00 . 

Recall that, the SDE in (18), with h{t,x.) G C'j’^([0,T] x satisfying 

dhg{t,x)/dt + Ct,xhe{t,x) = 0 in [0,r) x admits a weak solution xj* in [0,T). 
Further, let 7 ^ t, 0 < s < t < T, denote the transition semigroup for x^. Note that 
the extended infinitesimal generator associated with xj* is given by 

= A,x + a{t, x)D^i log h{t, x) • D^i , (32) 

where Ct x is the infinitesimal generator associated with xj (cf. equations (3) and 
(24)). 

Next, we assume that /: R""^ —>■ R has continuous partial derivatives up to the 
second-order which, along with /, vanish at infinity and satisfies 

^rM/(-) + <xr«,j(-) = o, o<s<t<r. ( 33 ) 

Then, we have the following result that characterizes the set of attainable distri¬ 
butions for the controlled-diffusion process associated with (30). 

Proposition 4.1. Given any admissible control {ut)o<t<T-e! then the attainable 
distributions associated with x“ (cf. equation (30) ) and that of xf* (cf. equa¬ 
tion (18) for each t G [0,T], are identical. 


^Such an optimal admissible control has been studied using the stochastic control arguments 
(i.e., the logarithmic transformations approach) from Fleming (e.g., see Fleming [11]; cf. Section 2). 
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Proof. Suppose that / satisfies (33). If we applying the Ito’s formula to Ts,tf{'^t) 
for 0 < s < t < T, then we obtain 

/(xn - ro,J(x“) = ^‘1 + mi(s,x“) . 

m 7X .| 

+ y^._ •i:)^i/(x“) +a(s,x^)u^ 

3 —^ 

+ \J^ tr(a(s,x“)i:)2^7;,t/(x“))}ds 

+ f a{s,^:)D,iTs,tf{^':)-dWs (34) 

Jo 

Note that (33) holds a.e. with respect to the Lebesque measure and that of As¬ 
sumption 1.2 (cf. Remark 1.3) ensures that the distribution law of x“, for each t, 
is absolutely continuous with respect to the Lebesque measure. 

Then, using (33), the first integral on the right-hand side of (34) a.s. equals to 

(a{s, x'f)us - a{s, x.'f)D^i log h{s, x“)^ • (35) 

which is integrable with respect to the underlying probability measure. Thus, if we 
take the expectations in the above equation (and noting that our choice of h which 
gives an admissible Markov-type control a{t,Xt)D^i log h{t,xt) (cf. Benes in [3] for 
related discussions)), then we obtain 


E{/(xr)}-E{/(x(')}=E|^^a(s,x:)w.-Z?,ir.,t/(x:)ds| 

a(s, x^)T) 3 ,i log/i(s,x^) ■ D^iTs,tf{^s)ds^ 

= 0. (36) 

The claim follows easily from this, which completes the proof of Proposition 4.1. □ 


4.2. Connection with stochastic control problems. Here, we provide a suffi¬ 
cient condition on the existence for the optimal admissible control associated with 
Problem 1. Let St be an operator, acting on the set of cr-finite measures on 
defined as follow 

^^(xt) = y'?(0,y,t,x)/r(dy), (37) 

where dSt^/dX is the Radon-Nikodym derivative with respect to the Lebesque mea¬ 
sure A and q{s,y,t,x) is the transition density associated with the SDE in (2). 

First, let us consider Problem 1 with a deterministic initial condition, i.e., when 
fj,o assumes a Dirac measure that is concentrated at a point f Then, we have 

the following result. 

Proposition 4.2. Suppose that p,o is a Dirac measure which is concentrated at a 
point ^ S Further, assume that 

i7(^T|5TMo) <+00 (38) 

and let h(t,x.) be given by 

h{t,x) = J q{t,x,T,z)-^^{z)dz, (t, x) G [0, T] X R"‘'. 


(39) 
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Then, = a{t,xt)Dj;i log h{t,Xt) solves Problem 1 with an optimal value of 

= H{p.t\StTo)- 


Proof. Note that 

h{0,xo) = J q{0,Xo,T,z)-^^—{z)dz 

= 1. (40) 

Recall that h{t, x) belongs to ([0, T] xR"'^) and satisfies dhs{t, x)/dt+Ct,xhe{t, x) 
0 in [0, T) X R""* and 

/i(T,x) = -^(x), xeR"'". 

aoT/^o 

Note that h{t, x*) is martingale and Xt)} = 1. Further, from Proposition 2.1, 

the SDE in (18) admits a weak solution xj* in [0,T] and then, by (20), x!f is 
distributed according to p-r- 

Let Px and P„^h be measures induced by xt and x(*, respectively, on the path- 
space (^([O, T]; R”‘^). Next, let us introduce the following change of measures 

^{i) = h{T,fT). (41) 

Then, we have the following 

E{log/i(t,xf)} = E{/i(t,Xt)log/i(t,xt)} 

< E{ h{T, Xt) log h{T, xt) } 

= H{pt\StPo), (42) 


where we used the fact that h{t,xt) log h{t,Xt) is a submartingale process.^ 
Next, let us introduce the following sequence of stoping times 

Tn = inf{s I |xs| > n} 

and 


Tn{oj) =T if |xs(a;)| < n} for every 0 <t <T. 

For t < r, if we apply Krylov’s extension of the Ito formula (cf. [19, section 10, 
pp. 121-128]), then we have the following 

E{/l(t AT„,XtAr„)log/l(t AT„,XtAr„)} < E / -11 u* 11 /i(s, Xs)ds. (43) 

Jo ^ 

Further, from the optional sampling theorem, we have the following 

E{h{t A Tn,XtA tJ log h{t A Tn,XtA tJ} < E{/l(t, Xt) log/l(t, Xj) } (44) 

and, by (42) 

E{/i(t,Xt) log/i(t,xt)} <4-00. 

On the other hand, t A Tn ^ t as n ^ oo, then, from (43) and (44), we have 

< E{h(t,Xt)log/i(t,xt)}. (45) 


^Note that (p{x) = x\ogx is convex and bounded from below. 
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Moreover, if we apply the Fatou’s lemma to the left-hand side of (43) (which gives 
us the opposite inequality), then we have 

/•i ^ 

E/ -||u*||^_i/i(s,Xs)ds = E{h(t,Xt)log/i(f,Xt)}. (46) 

Jo ^ 

Note that /i(f, x*) log h(t, xj) is a submartingale process. Then, using again the 
Fatou’s lemma and taking the limit t —>■ T, we have 

iJ(/rT|<STAto) = E{/i(T,XT)logh(T,XT)} 

= E^^i||u:||;_,ds. (47) 

If ut is any admissible control, then, using Girsanov’s transformation (e.g., see [14], 
[9] or [31]) and noting the fact that x^ is distributed according to fir, we have the 
following 


2ll t 


i = E{h(r,XT)iogh(r,XT)} 

= E |/i(r,XT)exp a~^{t,x^)ufdWt- 

> exp |e ^log/i(r,XT)-I- J ■ dWt - J 

= exp|i7(pr|5T/io)-E J . 

Hence, the above inequality further implies the following 

H{f,T\STf^o) <E ^\\u:\\l_,dt. 

This completes the proof of Proposition 4.2. 



(48) 


(49) 

□ 


Remark 4.3. Note that if Pxt Px^ are measures induced by X( and x“ on 
the path-space C([0,T],R”'’*). Then, using using Girsanov transformation (e.g., see 
[14]j, we can reinterpret the cost functional J{ut) in terms of the relative entropy 
between and P^,, i.e., 

id(PxH^xJ= J log^dPxr 

= -E|y a~^(t,yif)ut ■ dWt - J 

= J{ut). (50) 

Moreover, when the admissible control is optimal (i.e., u(. = a(t,Tit)D„,i logh(t,x.t)), 
we have the following 

dd(d^xrl^xJ = R(MT|5TM0) 

= J{ut), 
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which implies the global relative entropy is exactly equal to the relative entropy 
between the final measures px cind StPo- 

Note that, from Proposition 2.2, for pQ and px (with px SxPo), there exist 
two (T-finite measures uq and vx such that the statement in (28) holds. Letting 
/3 t(x) = dvx/dvo, then we have the following relations 

= Pt{^) J q{0,y,T,x.)vo{dy) (51) 


and 


^ = /'?(0,x,T,z)pT(z)dz. (52) 

dvo J 

For any initial random variable xq = ^ distributed according to po and satisfying 
f l^l'^dpo < + 00 , then we have the following result which is a generalization of 
Proposition 4.2. 

Proposition 4.4. Suppose that H{px\Sxi^o) < +00 and J(^dpo/diyo)dpo < + 00 . 
Let h{t,Ti) be given by 


h{t, x) 


J q{t,^,T,z)px{z)dz, 


{t,x) € [0,r] 


(53) 


Then, u* = a{t,Tit)D,^i log/i(t,Xj) solves Problem 1 with an optimal value of 

J{ut)=Ej^ ^\\ut\\l_,dt 

= H{px\Sxtyo) - HiPoWo)- (54) 

Proof. First, let us show that h{t,x) = Et_x{pT(xT)}- Note that, from (54), this is 
true if we show Et^x{pT(xT)} < +c», i.e.. 


]Et,x{pT(xT)} = y PTi^)dSxPo 

= j /Ot(x) q{0,y,T,x)dpo{y)^ dx 


9 ( 0 , y, T, x)px{x)dx dpofy) 


dpo 

dVQ 


dpo < + 00 . 


(55) 


Then, we can process as follow (cf. the proof part of Proposition 4.2). Let us 
introduce the following change of measures 


dP:^h ^ Pt(Ct) 
dPx h{0,fo)' 


( 56 ) 
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Then, we have the following (cf. equation (42)) 

E|log/i(<,x(‘)| = E I E{h{t, xt) log h{t, Xt) [xp} | 

<E{logMT,x$;)} 

= Hit^rlSriyo) ( 57 ) 

and in place of (43), we have the following 

IE I ^^Q^^^) E{h(t A Tn, XtAr„) log h{t A Tn, XtAT„) |xo}| - E|log /l(0, Xq) | 

r 1 1 'j 

ilKIII-'*'*'"-)*}- 

where 

E{log/i(0,xo)} = J log^d^io 

= H{htWo) < +00 

and, from Jensen’s inequality, we further have the following 

0 < [log < log [ ^l^duo < +CX:). 

J dh'o J duQ 

Then, using the limit arguments (i.e., the Fatou’s lemma) as in Proposition 4.4, we 
obtain the following 

nT 1 

^ oW'^tWl-id't = HinTlSrvo) - H{no\iyo). ( 59 ) 

Moreover, for any admissible control ut, then we have 

pT 

^ > i7(MT|<STPo) - 77(Aio|uo)- (60) 

Jo ^ 

This completes the proof of Proposition 4.4. □ 


Note that the conditions (i.e., i7(^r|iSTUo) < +oo and /(d/io/duo)<^Mo < +oo) 
under which Proposition 4.4 holds are rather difficult to verify. However, when 
fj-o has compact support, we can relax them with suitable conditions due to the 
following lemma. 


Lemma 4.5. Suppose that /io has compact support and H{plt\St tJ^o) < +oo. Then, 
we have 


H{p,t\St>^o) < +00 and J< +oo- 
Proof. Note that h(t,x.) is smooth and if fio has compact support. Then, we have 
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Moreover, 


H{^it\Stvo) = J log + J log 

= H{^T\ST^J'o) + J log 

Then, we have to show that f \og{dST/dSTVQ)d^T exists. 

Let us define log” (j) = max{— log0, O}. Then, from Jensen inequality, we have 
the following 


_dST^ 0 i . , - [ d^Q g(0,x,T,y) 

log -1—(y)=iog 

d^T J dvQ 


{dSxvo / dX){y) 


duo(x) 


< 


g(0,x,T,y) 
dvo^^^'J {dSTVo/dX){y) 


, -dfio, . 
log 3 —1^- 


(iuo(x). 


Hence, we have 
dST^J,o 


log" 


dun 


{y)d^J.T{y) < 


log ^(x)^ g(0,x,T,y)di/o(x)^ duT(y) 


= [ log ^duo 
J dvQ 


= J log d(0,x)d^o(x) < + 00 . 


Moreover, we have the following 


log 


dSri^o 

dSTVo 


dfiT < / log 


dSxfJ'O 

dSxi'o 


dfiT 


= log J h{T,y)dST^io{y) 

= J log/i(0,x)d^o(x) < + 00 . 


This completes the proof of Lemma 4.5. 


□ 


Remark 4.6. In Problem 1, we ean also include a state dependent term in the cost 
functional of (31), i.e., 

J{xf,ut) +«(x“)|dL 

where k is a nonnegative, real-valued continuous function on R"'^. 

Then, we can proceed in the same way as above if we take h{t,x.) in the ker¬ 
nel of the operator {d/dt + £t^x ~ k) and q{s,x.,t,y) as the fundamental solution 
of dq{t,x,T,y)/dt -\- £t^^q{t,x,T,y) - K{t,x)q{t,x,T,y) = 0 in [0,T) x and 
Vnxit-fT q{t,x.,T,y) = (5y(x) /or x, y € R”‘^. Moreover, h{t,x.) admits the following 
probabilistic representation 


h{s,x) = Es,x 


h{T, XT’) exp 


K{yit)dt 


dt. 


Note that, g(s,x, t,y) is the transition density of the killed diffusion process with 
the same drift and diffusion terms as that of (2) with killing rate (or potential) k 
( e.g., see [5] for additional discussions). 
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5. Remarks on the invariance property of the path-space measure. In this 
section, we briefly remark on the invariance property of the path-space measure of 
the diffusion process pertaining to the chain of distributed systems. Note that such 
an interpretation makes sense if the diffusion process {^t)o<t<T, which is associated 
with the SDE in (2), is considered as a random variable with values on a space 
of functions T]; containing its trajectories (e.g., see [32], [9] or [31]). As 
a result, we can determine local information about the measure induced by X[q 
F or example, for a given Lp G (^^([O, Tj; and small e > 0, we can provide an 
asymptotic estimate on the probability of a small er-tube around (^^([O, T]; R"'^)- 
function using 

as £ ^ 0, (61) 

where L{t, (p) is the Lagrange function given by® 


- ip\\ < e} Ke exp<^ - 


{-I 


p) 


M{t,(p) - p 


(62) 


Note that the above asymptotic estimate in (61) provides a probabilistic interpreta¬ 
tion for the most probable paths, i.e., the most probably trajectories that minimize 
the functional fj' L(t, (p, p)dt. Moreover, these extreme trajectories (which belong 
to (^^([O, Tj; R"'^)) are solutions to the following Euler-Lagrange differential equa¬ 
tion 

d d d 

—L{t,ip,p)-— — L{t,ip,p)=0. (63) 

The following result shows that adding a perturbation Ga{t, x)D^i log h{t, x) to the 
original drift term M(t, x) does not change the extreme trajectories of the diffusion 
process associated with the chain of distributed systems in (2). 


Proposition 5.1. Assume that M(t,xt) G C^{[0,T] x R”‘^;R'“^) and cr{t,xt) G 
C^([0,r] X R""^;R'^^'^),- and suppose that h{t,x) G C'jJ^’^([0,T] x R""^) is a strictly 
positive function that satisfies dh{t,x.)/dt + Ct,xh{t,x.) = 0 in [0,T) x R"‘*. Then, 
the diffusion processes Xt and x* with the same diffusion term (T(t,x) and whose 
drifts areM(t,x) and M(t, x)-|-G'a(t, xjU^,! log/i(t, x), respectively, have the same 
extreme trajectories. 


Proof. Note that we can rewrite the Euler-Lagrange differential equation associated 
with xt as follow 

+ a-^G^(M - p'ja-^G^D^M + - p'jcr-^G'^ 

= 0. (64) 

where denotes the vector derivative with respect to ip. 
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Similarly, for the Lagrange function associated with it, i.e., 


2 

a-i’ 


(65) 


we can further compute the associated Euler-Lagrange differential equation as follow 

(m - D,^a-^G^ (m - pj 

+ a-^G^(M - p^a-^G^D,^M + - <^)a"^G'^ 


d h ^ ^ Th -1 


+ aG'^D^p log hD^^a-^G'^ (M-p'^+ aG"^ log h a-^G^D^^M. 

+ ^cr“^G^ + (tG^D^p log D^aG^Dp log h 

+ (cr-^G^ (m - (/;) + aG'^Dp log aG^D^ log h 
,rp dDp, log h 


da- 


o 

ot 
= 0 . 


at 


( 66 ) 

Note that the first five terms in (66) are identical to the Euler-Lagrange differential 
equation associated with L (cf. equation (64)). Further, noting that the last six 
terms in (66), i.e., 

aG'^D,p log hDpa-^G^ (m-p'J+ aG'^D^, log h a-^G'^D^.M 
+ a~^G"^ — i^ Dp,aG^D,p log h + (^aG^ log D,paG"^D,p log h 

'(m-P^ aG^Dl log h + (aG^Dp, log h) aG'^D^ 


-l/^T I 


+ a-^G 
da 


+ Dp, log h + aG 
dt 


rp dDp, log h 
dt ’ 


with additional steps, reduced to the logarithmic transformation of h that satishes 
the following (cf. equation (10)) 

i91og h 


at 


+ M.Dp, log/i -I- i tr^aG^L?^ log/i^ - log= 0. 


(67) 


Hence, we see that the above two Lagrangians (i.e., the Lagrange functions in (62) 
and (65)) yield the same Euler-Lagrange equation. The claim follows easily from 
this, which completes the proof of Proposition 5.1. □ 
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